WebIDL tests #271
WebIDL tests #271
Conversation
The named property visibility algorithm should cause expandos to shadow named properties, not the other way around.
|
How long do you expect making followups to take? At the very least the files need to be in the right place in the tree. And you don't want to loose the review comments. Is there anything really substantive to fix? |
|
There are additional comments in https://critic.hoppipolla.co.uk/r/247 |
…t.width to HTMLOListElement.start)
|
This PR has become kinda stale and it's not clear at this point who it's waiting on action from. @plehegar? |
|
@sideshowbarker @plehegar - my understanding is that @ylafon agreed to address all of the comments for this PR although I don't know if his plan is to update the PR or to submit a new one. Yves - what is the plan and timeframe to address this PR? |
|
@tobie: You are working on WebIDL, right? What do you want to happen with these tests? |
|
Note that there is a PR on this PR, comments on this one were addressed in https://github.com/ylafon/web-platform-tests/commits/submission/heycam/WebIDL-tests-1 |
|
@jgraham: wow. I wasn't aware this test suite existed. I'm debating the value of reviewing and maintaining such a test suite compared to investing this time improving idlharness in both depth (keep the harness in sync with spec changes, adding more tests to it) and breadth (e.g. adding it to the +50% of specs in wpt who don't have idlharness tests yet, keeping the existing ones up to date, etc.). @domenic, @annevk, @bzbarsky, @heycam: would love your thoughts on this. |
|
It's not clear to me why we dropped the ball here. These look like good tests. Someone should run them in browsers to see if they're reasonably accurate and then just land them (using the branch from @ylafon as that seems to contain all the fixes). |
I'm not debating this. However, given constrained resources (which is clearly demonstrated by the fact no one had enough cycles to follow through on merging those), I'm wondering:
|
|
@Ms2ger might be able to judge whether they offer something in addition to idlharness. Maintenance will generally fall out of implementations finding that tests start failing or passing. |
|
@gsnedders your input would also be valuable here. |
In doing that, don't we loose the ability to help implementors fix their WebIDL bindings by maintaining the tests ourselves? |
|
On irc, @Ms2ger said:
|
Sure, but as you said, you can't have it all. |
|
My personal experience with modifying idlharness is that it's a huge pain. The code is scattered all over, there's lots of repetition, no particular rhyme or reason to where certain things are tested, etc. Also, idlharness assertions are not that well factored into test() blocks, so it's easy to get no coverage of all sorts of stuff because you don't implement some one thing somewhere that has nothing to do with the other stuff idlharness is testing. Fixing these problems is a pretty huge undertaking. So I actually somewhat favor less monolithic, more maintainable tests to some extent. In principle, I'm not sure anything prevents us from rolling these tests into idlharness and ensuring that idlharness runs on interfaces that exercise the relevant functionality. In practice, let's consider the workflow around adding a test for a new IDL feature. With small tests like this, you pick an interface that uses the feature, write a test against it. With idlharness, you pick an interface that uses the feature. You make sure idlharness is running against that interface. You make sure the idlharness parser can parse the feature (want to test And that's assuming you can do it at all. If you have some function that exercises the feature you want, and it needs some specific arguments, auto-figuring-out how it gets those arguments is quite nontrivial. Can they be constructed via constructor calls based on their IDL interface types? What if those have no constructors but we have factory methods? etc. As a concrete example, how would you write a [Clamp] test in idlharness? You need to observe the value that gets passed through to the underlying implementation, but you need to know something about the underlying implementation to manage to do that, right? I guess the upshot of all that rambling is that I think we should definitely get targeted tests of specific features written, outside of idlharness. idlharness is then somewhat useful for checking that the IDL in browsers matches the spec IDL, effectively, for the things idharness can test... |
|
All true, and certainly it's not possible to test all of WebIDL's requirements with idlharness alone. I do think that it is also valuable for things that can be tested in idlharness, to be tested there. |
|
Sure. I think it's more important to have tests than anything else. So what I think we should do in this concrete instance is get these tests landed, then see about porting those of them that can be ported to idlharness. |
ea389ca278 chore(package): bump version number to 20.0.1 518e5cc12b chore(CHANGELOG): regenerate aa3588b457 chore(dist): update 7ceaf0ee5b refactor(lib/webidl2): enum as a module (#318) 27b0eb31d3 fix(lib/webidl2): emit error message correctly (#317) bcb8c32c69 docs(lib/writer): fix extendedAttribute example 1ddd771ed8 docs(lib/writer): document generic() (#315) 86029c5866 chore(package): bump version number to 20.0.0 f195b986b8 chore(CHANGELOG): regenerate 7423f5d8f7 chore(dist): update 996d4f7e71 feat(lib/writer): introduce generic() hook (#314) 87b07ec705 refactor(lib/webidl2): modularize Default (#312) be140dad86 refactor(lib/webidl2): modularize Token and list() (#311) 464d3a1e2b refactor(lib/webidl2): modularize Includes (#310) 6b3b7c54f5 chore(package): bump version number to 19.0.1 c68baccbb2 chore(CHANGELOG): regenerate 97851631ab chore(dist): update 1cc6b3e257 fix(lib/writer): call ts.trivia with actual string (#309) e84ba4fd34 chore(CHANGELOG): regenerate 37f87e6393 chore(package): bump version number to 19.0.0 6486380999 update dist 6340c3e1e4 fix(lib/webidl2): restore error fields (#308) c6a7c97358 fix(lib/webidl2): rename float as decimal (#307) 048e3d748e BREAKING CHANGE: remove all trivia fields (#306) f37e39e928 BREAKING CHANGE: remove baseName and escaped* fields (#305) b0c5a9e5e6 BREAKING CHANGE: remove token fields (#304) e1aa18822f BREAKING CHANGE: remove trivia from boolean fields (#303) 112862924c refactor: merge error functions (#299) 1926437649 fix(checker): add an empty line between messages (#298) 3de7e3066a chore(package): bump version number to 18.0.1 a3637e9c20 update build 09c4e4f725 chore(package): bump version number to 18.0.0 df4c73deef fix(lib/validator): validate when overloads are all from mixins (#296) a8e81d479e refactor: use module for checker (#295) 679395293f fix(checker): use webpack build result (#294) abd41222d5 refactor: use ES module syntax and bundle by webpack (#293) aa75ae538c feat: add validation feature to the checker page (#292) b7e7b1ba99 feat: a validator for IDL (#262) 852194f5ee refactor: remove remaining JSON idlTypes (#291) b8590cd281 refactor: use Type for primitive types (#290) 44987f3bf5 refactor: generic type as a class (#289) c86dac1295 refactor: union type as a class (#288) 64f0fa50d7 refactor: argument as a class (#287) 209c1d4cf4 fix(lib/webidl2): disallow empty generic type (#286) 927328d5c7 fix(lib/webidl2): disallow null on constants (#284) 45287bf978 refactor: use list() in identifiers()/enums (#285) 2d555baf51 refactor: extended attributes as a class (#283) 11afb8e7ea refactor: extended attribute as a class (#282) 7d1562c406 refactor: default value as a class (#281) 812a292252 BREAKING CHANGE: constant as a class (#280) 44b68f8afc refactor(lib/writer): use tokens object in container() (#273) a9bb14d43e add a test case (#277) 7c07ca4eb5 refactor: attribute as a class (#276) a25f052736 refactor: operation as a class (#275) dba7a4b8b7 test(lib/webidl2): nullable constant with typedef identifiers (#279) 903c374ebb refactor: callback function as a class (#278) f396b5182d refactor: iterable as a class (#274) 810d2db4cf refactor: containers as classes (#272) ce9787caab refactor: namespace as a class (#271) 56a590d327 refactor: dictionary as a class (#270) 5e51ccec85 refactor: field as a class (#268) 9643425b8e refactor(lib/webidl2): move toJSON to Definition (#269) c13e3aa65f refactor: enum as a class (#267) ae4a1b4fd8 refactor: typedef as a class (#266) 81edfc0774 refactor: includes as a class (#265) f9c0d44f49 chore: add .gitattributes (#264) 51618dced6 style: apply no-trailing-spaces (#263) d9c160f4ba chore(package): bump version number to 17.0.2 1a2147bb20 fix(lib/webidl2): apply a preceding hyphen for identifiers (#261) c27f3c3e94 fix(checker): avoid using innerText (#260) 4b0443568c fix(lib/webidl2): restore enum value string form in error (#258) 805e2b2ea2 docs: lowercase the example namespace "Console" (#257) 16d5a92b2d chore(CHANGELOG): regenerate 3bb235ca95 chore(package): bump version number to 17.0.1 1aa1258028 fix: unescape includes/idlTypes/inheritances (#253) e57f46632b fix(lib/webidl2): prevent any in a union type (#254) 8cbec8b4d0 Revert "fix(lib/webidl2): prevent any in a union type" 73c15634ac fix(lib/webidl2): prevent any in a union type c393451a43 feat(lib/webidl): subclass standard error (#247) 6e2a83bfee chore(CHANGELOG): regenerate 3f8ac3f23b 17.0.0 9e2c43da44 BREAKING CHANGE: remove .extAttrs from arguments (#248) ec05bc243b 16.1.0 4ae64a8766 Add unit tests for writer template functions (#245) dd89780e22 feat(lib/webidl2): better error messages (#244) edaf314b6a Add docs for template feature (#243) df4484597e Revert "add test for writer template feature" 19f5e6e8f8 Revert "upgrade eslint" 7e886e9f26 add test for writer template feature 1313c80c16 upgrade eslint 1014b33145 feat(lib/writer): write() with optional templates (#241) 5d5a645936 chore(package): bump version number to 16.0.0 3c538be29f BREAKING CHANGE: merge modifier fields (#240) 7350bb1227 refactor(lib/writer): use container() (#239) b215543038 refactor(lib/writer): use token() (#238) 0a0bfa9ee9 refactor(lib/writer): reduce conditional constructs (#237) 40a048b800 chore: remove jscov (#236) 018b24f7fc chore(package): update eslint (#235) d38c4ed618 eslint: prevent eval() (#234) 8ff7a963cf chore(package-locK): regenerate 731f5edb5a chore(CHANGELOG): regenerated d759b8c968 chore(package): update version + deps 568e1d5db6 docs(README): enum value's type is now enum-value (#233) f3462f9501 BREAKING CHANGE: enum values should be of type enum-value (#232) 46b250286d fix: escape top identifiers (#229) 91bba911e5 fix(lib/webidl): disallow multiple special keywords (#224) b2a9f6c901 BREAKING CHANGE: Drop support for implements statement (#106) da93c9373b refactor(lib/writer): horizontally shorter type() (#221) 2e03190e30 chore(README): operation body always has idlType (#218) 6cc3277c9d chore(package): bump version number to 14.0.1 a3e89c87d2 fix(index): import writer from index (#219) 1582972c21 chore(CHANGELOG): regenerate 402e4e3514 chore(package): bump version number to 14.0.0 698b8e991d Document changes for idlType and extAttrs (#217) 7328816495 Document trivia for root type declarations/members (#214) b3fd89040d BREAKING CHANGE: remove support for legacyiterable (#216) cce0d6d484 Document trivia for iterable-likes (#213) b5b4d90a12 Document trivia for interfaces/mixins/namespaces (#212) 69a0f2b5ac Document generic array, escapedName and removal of sequence (#211) 7d08a0c8af BREAKING CHANGE: support full whitespace conservation (#185) bbe95578bd trivia for implements/includes (#209) 9df331ce7d Add trivia for enums and typedefs (#207) 2559b95c75 Add trivia for callbacks (#206) 4cbd4143ba Add trivia for partial types (#205) fc147026f7 trivia for dictionaries (#204) be9126fa7b Add trivia for mixins and namespaces (#203) b02aa3366a Use eslint minimally (#202) 74cd53340d Add trivia for extended attributes (#201) 7524981417 Add trivia for extended attribute identifiers (#200) 3d682826e2 Add trivia for const member type (#199) 25d5919278 trivia for arguments (#198) b17417ca83 trivia for operation (#197) 6e12a109d1 trivia for inheritance (#195) 79d791a389 Add trivia for iterable declarations (#194) 0cb3e7cba4 Add trivia for modifiers (#193) ecebf30070 Add trivia for idlTypes (#192) 32f11bc730 BREAKING CHANGE: remove nullable field from const type (#189) f4292a7a21 Generics always as an array (#188) 1ea9f9c91a fix: prevent empty iterable declaration (#187) d36f7e4b7e Add trivia field for interfaces/mixins (#186) git-subtree-dir: resources/webidl2 git-subtree-split: 306cf6479f909a38297c4982e79171958669fe55
Here are the rest of the tests I have for Web IDL v1.